1. Field of the Invention
The present invention relates to a video packet generating method, a video decoding method, a media multiplexer, media demultiplexer and a multimedia communication system which are used in the process of sending coded audio, video (moving image) and data packets as a multiplexed stream from the transmitting side and demultiplexing the multiplexed stream into audio, video and data at the receiving side. Furthermore, the present invention also pertains to a bit stream converter which permits effective conversion of an error resistance syntax according to the channel condition.
2. Description of the Prior Art
To facilitate a better understanding of the present invention, a description will be given first of the case where a coded video stream encoded by a conventional video coding system, for example, the ITU-T Recommendation H. 263 (hereinafter referred to as H. 263), is sent after being multiplexed by the ITU-T Recommendation H. 223 (hereinafter referred to as H. 223). H. 223 defines a multiplexing system by which compress-coded audio, video and data streams are multiplexed into one bit stream for transmission.
FIG. 1 depicts the configuration of an H. 233-recommended multiplexing part. As shown, the multiplexing part comprises two hierarchies of an adaptation layer and a multiplexing layer. The adaptation layer input thereto, as packets called AL-SDU (Adaptation Layer Service Data Unit), coded streams from an application layer that encodes speech and video, and adds each packet (AL-SDU) with an error correcting code (CRC) and other necessary control information (AL-PDU (Adaptation Layer Protocol Data Unit). The multiplexing layer inputs thereto, as packets called MUX-SDU (Multiplex Service Data Unit), the packets (AL-PDU) of various media from the adaptation layer, and multiplexes them into one bit stream for transmission. The bit stream multiplexed in the multiplexing layer is called MUX-PDU (Multiplex Protocol Data Unit), and comprises a sync flag, a header and information fields for storing the packets (MUX-SDU). Incidentally, the method of dividing the coded streams from the application layer into packets (AL-SDU) for processing in the multiplexing part is outside of the scope defined by H. 223.
The above-mentioned multiplexing part is a function at the transmitting side. The receiving side has a demultiplexing part whose function is the inverse of the multiplexing at the transmitting side. In the demultiplexing part the bit stream (MUX-PDU) received in a demultiplexing layer is demultiplexed into packets of various media (MUX-SDU), which are output to the adaptation layer (AL-PDU). In the adaptation layer the error detecting code (CRC) contained in the packet (AL-PDU) is decoded and a check is made for an error in the received packet (AL-PDU). The result of error detection is output to the application layer together with the coded media information stream (AL-SDU) in the packet. It is outside the scope defined by H. 223 how to use the result of error detection in the application layer.
By the way, the video coding system by H. 263 encodes the input image into blocks of a predetermined size (each of which is comprised of a luminance signal of a 16 by 16 pixel size and a color difference signal of an 8 by 8 pixel size and is called a macroblock); it is considered that the coded video stream by H. 263 may be rendered into the packet AL-SDU, for example, by a method of combining piece of coded data of plural macroblocks into one packet.
As referred to above, however, it is recommended by H. 223 that the error detecting code be added for each packet AL-SDU and that the application layer at the receiving side receive the result of error detection together with the packet AL-SDU. Hence, when pieces of coded data of plural macroblocks are combined into one packet AL-SDU, it can be known from the result of error detection that there is an error in any one of the plural macroblocks contained in the packet AL-SDU, but it is impossible to specify the macroblock in which the error has occurred and the information in which the error has occurred. Accordingly, the prior art fails to practice error concealment by making effective use of the result of error detection received in the application layer.
Next, a description will be given of problems of the moving image communication between circuits of different error rates.
With the recent broadening the band of a digital transmission line and development of multimedia communication technology, it has become feasible to transmit moving image signals over various transmission lines. In the field of communication there is in widespread use a teleconference/videophone system that uses the H. 261 moving picture coding system on the ITU-TH. 320 terminal intended for connection to ISDN. Further, at the H. 324 terminal assumed to be connected to an analog public network, there are supported not only the H. 261 moving image coding system but also the H. 263 coding system which is higher in coding efficiency than the former coding system.
Moreover, there is recommended, as a terminal assumed to be connected to an IP network such the Internet, H. 323 that supports H. 261/H263 as is the case with H. 324. Accordingly, since at such wired system terminals there is supported a set of limited moving image coding systems based on ITU Recommendations, the interconnection of different protocols between terminals for limited use is guaranteed to some extent.
On the other hand, IMT-2000 is now under study by ITU as a standardization of a multimedia communication system using radio channels, and MPEG-4 of ISO is regarded as a likely moving image coding system. The diffusion of present portable terminals (such as PDC and PHS) are extremely wide, and if multimedia terminals based on IMT-200 comes into use as substitutes for existing portable telephones, an enormous demand therefor is expected. In such a case, the interconnection between the new multimedia terminals and existing wired system multimedia terminals is important, but this is the interconnection between channels of different error rates, not the interconnection between the existing wired-system circuits.
Predicated on the use with radio channels, MPEG-4 contains a function of increasing the error resistance at the video data level; there has not been established so far a technique for effectively converting the function at the time of interconnecting the multimedia terminals.
The conventional bit stream converter has such a construction as described above, and hence it has a defect that an error resistance syntax cannot effectively be converted between the moving image coding system used in a channel of low error rate and the moving image coding system used in a channel of high error rate according to channel conditions.